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Abstract 

Today’s  military  electronic  infrastructure  solves  many  problems  while 
creating  others.  Using  computers,  battlefield  and  global  awareness  is  brought  to  bear 
through  the  near  real-time  linking  of  sensor  platforms  from  around  the  globe.  These 
intelligence  networks  produce  vast  amounts  of  data  that  must  be  parsed,  interpreted, 
digested  and  stored  by  information  gathering  systems.  As  the  amount  of  intelligence 
data  continues  to  increase,  these  text-based  systems  become  cumbersome  and 
inadequate.  To  ensure  vital  information  is  not  overlooked  or  discovered  too  late, 
other  forms  of  intelligence  product  management  and  data  navigation  need  to  be 
investigated. 

This  thesis  explores  procedures  for  enhancing  the  capabilities  of  Project 
Broadsword,  an  intelligence  data  retrieval  system,  using  three-dimensional  data 
visualization.  Employing  a  stand-alone  representative  environment,  this  research 
develops  methods  and  techniques  for  overcoming  problems  encountered  when 
visualizing  large  quantities  of  data.  Textual  handling  through  graphical  triggers  is 
also  addressed. 

The  results  of  this  research  demonstrate  the  effects  of  utilizing  three- 
dimensional  visual  cues  to  organize,  perceive,  and  navigate  vast  amounts  of 
intelligence  products  and  their  associated  metadata.  Conclusions  drawn  from  this 
research  directly  affect  the  next  major  release  of  the  Project  Broadsword  system, 
currently  under  development. 
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THREE-DIMENSIONAL  DATA  VISUALIZATION  OF  ELECTRONIC 


MILITARY  INTELLIGENCE  USING  THE  PROJECT  BROADSWORD  SYSTEM 


1.  Introduction 


1.1  Overview 

Information  is  power.  This  notation  has  been  the  crux  of  successful  political 
and  military  actions  throughout  recorded  history.  Measures  of  this  power  range  from 
the  influence  of  shamans  and  priests  who  were  well  versed  in  the  ways  of  the  natural 
world  to  General  Schwartzkoff  with  his  sophisticated  network  of  surveillance 
platforms  during  the  Gulf  War. 

Today’s  military  decision-makers  utilize  data  and  information  under  the 
concept  of  Command,  Control,  Communications,  Computers  and  Intelligence  (C4I). 
The  concept  of  “Command”  encompasses  all  levels  of  authority  from  the 
commander-in-chief  to  the  airman  in  charge  of  a  security  detail.  Each  level  requires 
the  key  individual  to  receive  and  assess  known  situational  information,  build  options, 
and  then  execute  his/her  final  decision.  According  to  the  Department  of  Defense’s 
Dictionary  of  Military  and  Associated  Terms.  “Control”  is  defined  as 

That  authority  exercised  by  a  commander  over  part  of  the  activities  of 
subordinate  organizations  or  other  organizations  not  normally  under  his 
command,  which  encompasses  the  responsibility  for  implementing  orders  or 
directives.  All  or  part  of  this  authority  may  be  transferred  or  delegated. 
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In  this  context,  “Control”  lends  itself  to  the  means  by  which  information  and 
command  functions  are  passed.  “Communication”  depicts  the  means  by  which  full- 
duplex  information  is  moved.  In  essence  this  can  be  anything  from  runners  and 
carrier  pigeons  to  fiber  optic  network  lines.  Today’s  decision-makers  no  longer  rely 
on  pulp  media  as  their  sole  source  of  data  and  information.  With  the  introduction  of 
the  first  computer  by  John  V.  Atanasoff  in  1939,  the  need  for  and  use  of  “Computers” 
has  grown  rapidly.  Through  this  electronic  realm,  we  are  able  to  collect,  manage, 
cross-reference,  validate,  meld,  and  examine  intelligence  even  as  the  amount  of  data 
rises  exponentially.  Finally,  “Intelligence”  is  the  collection,  analysis,  presentation, 
and  utilization  of  information  on  an  area  or  subject  of  interest. 

As  identified  by  Lee  Paschall,  former  Director,  Defense  Communications 
Agency,  the  line  between  information  needs  and  information  overdose  is  a  blurry  one 
that  is  difficult  to  define  (1).  Numerous  Department  of  Defense  programs  and 
research  efforts  have  been  put  in  place  to  try  and  calculate  the  needs  of  the  decision¬ 
makers  while  understanding  that  human  factors  play  a  vital  part  in  the  equation. 

This  research  examines  the  “I”  in  C4I,  with  respect  to  the  needs,  productivity, 
and  natural  ability  of  the  intelligence  analyst,  through  interaction  with  an  intelligence 
data  dissemination  effort  sponsored  by  the  Air  Force  Research  Laboratory  (AFRL), 
Rome  Research  Site.  Known  as  Project  Broadsword,  this  web  technology-based 
production  level  program  is  enhanced  utilizing  data  visualization  techniques. 
Conclusions  are  drawn  about  whether  the  added  visual  cues  increase  the  intelligence 
analyst’s  technical  productivity  while  maintaining  an  easy  to  adapt  and  use  computer 
interface. 
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1.2  Background 


In  1994  the  Secretary  of  the  Air  Force,  Dr.  Sheila  Widnall,  and  Chief  of  Staff, 
General  Ronald  R.  Fogleman,  directed  the  Air  Force  Scientific  Advisory  Board 
(SAB)  to  identify  those  technologies  that  will  guarantee  the  air  and  space  superiority 
of  the  United  States  in  the  21st  Century  (2).  To  that  end,  a  study  was  conducted  in 
1995  that  identified  those  areas  that  are  crucial  for  the  United  States  to  continue  its 
dominance  of  air  and  space.  The  final  report  emphasized  the  importance  of  a  global 
awareness  capability  for  the  warfighter.  To  meet  this  challenge  head-on,  Project 
Broadsword  was  created  at  the  AFRL,  Rome  Research  Site  Information  Directorate  in 
New  York  State. 

The  main  thrust  of  Project  Broadsword  is  to  allow  the  intelligence  analyst 
seamless  access  to  vast  amounts  of  heterogeneous  data.  Achieving  this  goal  requires 
varied  layers  of  information  dissemination  and  presentation  through  all  levels  of 
command.  To  efficiently  solve  such  real-world  problems  is  one  of  the  major 
challenges  facing  the  Air  Force  today.  As  stated  in  New  World  Vistas,  “The 
problems  of  the  next  decade  are  to  identify  the  relevant  databases,  to  devise  methods 
for  collecting,  analyzing,  and  correlating  them,  and  to  construct  the  needed 
communication  and  distribution  architectures.”  (3) 

1.3  Problem  Statement 

Since  the  advent  of  the  digital  computer  in  1939  the  amount  of  data  and 
information  available  to  an  intelligence  analyst  has  grown  exponentially.  Digital 
transfer  of  intelligence  provides  knowledge  faster,  in  bulk,  and  allows  for  more 
precise  tactical  decisions.  Today’s  military  intelligence  products,  which  can  be 
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found  in  text,  multimedia,  and  image  form,  are  navigated  and  exploited  through 
textual  interfaces.  As  the  amount  of  intelligence  data  continues  to  increase,  these 
interfaces  become  cumbersome  and  inadequate.  In  order  to  ensure  vital  information 
is  not  overlooked  or  discovered  too  late,  other  forms  of  product  management  and  data 
navigation  need  to  be  explored.  Addressing  the  SAB’s  Global  Awareness  agenda, 
Project  Broadsword  has  developed  a  universal  textual  interface  for  data  source 
management  and  has  begun  to  explore  the  potential  benefits  of  data  visualization 
using  a  two-dimensional  approach. 

1.4  Research  Objectives 

The  primary  purpose  of  this  research  effort  is  to  create  a  representative 
environment  to  examine  ways  by  which  Project  Broadsword  can  offer  management  of 
intelligence  products  through  data  visualization.  Based  on  verbal  request  of  those 
attending  the  1999  spring  Project  Broadsword  user  conference,  this  research  is 
directed  towards  visualization  in  the  three-dimensional  realm.  Key  areas  of  interest 
include  product  representation  utilizing  three-dimensional  glyphs  as  well  as  suitable 
and  effective  navigational  approaches.  To  fulfill  this  objective,  three-dimensional 
visual  cues  must  offer  the  analyst  an  efficient  and  effective  way  of  organizing, 
understanding,  and  navigating  vast  amounts  of  intelligence  products  and  their 
associated  metadata.  The  Project  Broadsword  verification  and  validation  team  holds 
the  principle  role  of  determining  the  suitability  and  productiveness  of  this  approach. 
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1.5  Assumptions 


If  the  results  of  this  thesis  are  to  be  integrated  into  future  Project  Broadsword 
baselines,  a  level  of  backwards  compatibility  must  be  maintained  to  the  version  3.0 
release.  Additionally,  all  work  is  constrained  to  the  environment  supplied  within  the 
Project  Broadsword  development  and  integration  vault.  This  is  to  fulfill  functional 
requirements  and  maintain  primary  security  certification. 

1.6  Scope  and  Limitations 

This  research  effort  explores  an  effective  and  efficient  three-dimensional 
visual  representation  of  user  requested  products  when  utilizing  the  Project 
Broadsword  interface.  This  research  narrows  its  scope  to  a  limited  user  base  and 
provides  a  proof-of-concept  environment  only.  No  attempt  has  been  made  to  produce 
a  production  quality  entity  or  provide  tailored  features  to  a  particular  Project 
Broadsword  user  class. 

1.7  Research  Summary 

This  research  explores  data  visualization  techniques  that  enhance  an  analyst’s 
exploitation  capability  when  employing  the  Project  Broadsword  system.  Using  a 
stand-alone  representative  environment,  this  research  develops  methods  and 
techniques  for  overcoming  problems  encountered  when  visualizing  large  quantities  of 
data.  Textual  handling  through  graphical  triggers  is  also  addressed.  The  results  of 
this  research  demonstrate  the  effects  of  utilizing  three-dimensional  visual  cues  to 
organize,  perceive,  and  navigate  vast  amounts  of  intelligence  products  and  their 
associated  metadata. 
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1.8  Document  Summary 


The  remainder  of  this  document  is  divided  into  six  chapters.  Chapter  2 
discusses  relevant  issues  in  intelligence  and  data  visualization.  This  chapter  also 
explores  the  Project  Broadsword  solution  to  seamless  intelligence  exploitation  from 
dissimilar  heterogeneous  data  sources.  Chapter  3  presents  the  research  process  and 
design  methodology.  Chapter  4  describes  the  research  development  environment. 
Chapter  5  outlines  design  and  implementation  decisions  made  based  on  data 
visualization,  data  navigation,  and  software  development.  Chapter  6  contains  findings 
and  reviews  the  observations  made  during  this  research.  Finally,  Chapter  7 
summarizes  this  research,  identifying  specific  contributions  and  proposed 
recommendations  for  future  work. 
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2.  Intelligence  and  Data  Visualization:  Relevant  Issues 


2.1  Overview 

The  exploitation  of  military  intelligence  using  data  visualization  is  a  relatively 
new  technique  that  few  implementations  have  developed  and  put  to  practical  use. 

This  chapter  begins  by  providing  a  cursory  review  of  the  need  for  military 
intelligence.  It  then  introduces  Project  Broadsword,  a  software-based  data  retrieval 
system  that  is  able  to  access  multiple  intelligence  product  domains  through  a  single 
user  interface.  Finally,  elements  of  data  visualization  that  directly  effect  this  research 
effort  are  presented. 

2.2  The  Need  for  Intelligence 

The  basic  function  of  intelligence  is  to  provide  the  requisite  support  for 
timely,  sound  decisions  of  all  matters  both  in  and  out  of  a  military  conflict.  As  stated 
by  William  E.  Colby,  “The  purpose  of  intelligence  is  to  help  you  act  so  that  you  can 
have  a  better  rather  than  a  worse  future.  And  if  you  act  intelligently,  and  cause  a 
change  in  that  future,  then  of  course  the  prediction  turns  out  to  be  wrong — for  the 
right  reason,  and  you’ve  really  capitalized  on  what  intelligence  is  all  about.”  (4) 

Intelligence  is  performed  in  a  cycle  that  consists  of  collection,  production, 
and  dissemination  of  information.  With  this  in  mind,  the  tasking  laid  before  the 
analysts  is  to  understand  and  adapt  to  the  operational  environment.  Information 
about  the  environment  must  be  collected,  analyzed,  and  finally  molded  into  a 
comprehensible  output  for  use  by  a  decision-maker.  The  importance  of  this  process 
cannot  be  overstated.  As  emphasized  by  General  Ronald  R.  Fogleman,  the  ability  to 
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collect  information,  rapidly  correlate  it,  and  then  quickly  disseminate  it  to  the  users 
and  commanders  changes  the  nature  of  warfare  (1).  As  we  approach  the  21st  century, 
we  teeter  on  the  ability  to  locate,  track,  and  catalog  anything  on  the  face  of  the  Earth 
in  near  real  time.  This  capability  brings  with  it  an  ever-increasing  demand  on  the  part 
of  the  analyst  to  understand  and  ingest  more  data  at  a  faster  rate.  When  dealing  with 
these  vast  amounts  of  data,  we  must  control  the  flow  such  that  only  the  important  or 
key  elements  are  brought  forth  and  presented.  Thus,  only  what  is  immediately 
important  is  driven  to  the  attention  of  the  analyst.  Based  on  these  concerns, 
intelligence  exploitation  systems  are  continually  exploring  advanced  data  presentation 
techniques  to  maintain  equilibrium  between  our  need  to  get  more  information  faster 
and  our  ability  to  handle  it. 

2.3  The  Project  Broadsword  Solution 

Project  Broadsword  is  an  Air  Force  Research  Laboratory  effort  sponsored  by 
the  Joint  Intelligence  Virtual  Architecture  (JIVA),  the  National  Imagery  and  Mapping 
Agency  (NIMA)  and  the  Air  Staff.  In  performance  of  their  duties,  the  intelligence 
analyst  has  four  types  of  information  to  collect  and  exploit  -  imagery,  video,  text,  and 
audio.  The  mission  of  Project  Broadsword  is  to  make  retrieval  of  these  product  types 
as  seamless  as  possible.  In  order  to  accomplish  this  feat,  a  set  of  tools  and  services 
are  employed  which  allow  an  analyst  to  search  and  retrieve  information  from  a 
collection  of  heterogeneous  data  sources.  These  sources  include,  but  are  not  limited 
to,  the  Military  Intelligence  Database  (MIDB),  the  Image  Product  Library/Image 
Product  Archive  (IPL/IPA)  and  the  Automated  Message  Handling  System  (AMHS). 
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Project  Broadsword  is  made  up  of  two  elements:  the  client  and  the 
Gatekeeper.  The  client  provides  universal  access  to  all  available  intelligence  data 
sources  via  the  Gatekeeper  by  way  of  standard  Internet  protocols.  This  approach 
allows  the  analyst  to  use  a  common  web  browser  for  system  access.  The  Gatekeeper, 
a  robust  thin  layer  of  software,  performs  a  variety  of  internal  functions,  most  notably 
the  ability  to  simultaneously  query  multiple  dissimilar  intelligence  databases  while 
returning  uniform  results.  These  results  are  available  in  a  multitude  of  formats,  to 
include  text  listings,  a  chronological  timeline,  and  a  two-dimensional  visual  display. 

Results  are  presented  in  one  of  three  ways.  First,  all  results  may  be  returned 
as  textual  descriptions  as  seen  in  Figure  2.1,  with  accompanying  thumbnail  images 
(where  applicable)  similar  to  today’s  modem  commercial  Internet  search  engines. 
Figure  2.1  shows  a  product  with  a  thumbnail  image  and  the  data  elements  under 
which  it  was  cataloged.  These  data  elements,  known  as  metadata,  consist  of 
descriptions  that,  either  individually  or  when  combined,  uniquely  identify  a  product. 
Figure  2.1  displays  only  a  small  sample  of  the  possible  metadata  elements  available 
for  a  product. 

If  textual  descriptions  are  not  desired,  a  timeline  may  be  chosen,  as  seen  in 
Figure  2.2,  which  displays  all  products  in  chronological  order  of  creation.  This 
method  organizes  products  by  date  and  allows  a  larger  number  of  products  to  be 
displayed  in  the  same  amount  of  screen  space.  The  results  are  limited  to  thumbnail 
images  coupled  with  a  hypertext  link  to  their  associated  metadata.  Selecting  a 
hypertext  link  generates  a  window  containing  the  selected  product  metadata,  similar 
to  Figure  2.1.  Figure  2.3  shows  the  third  presentation  method,  a  two-dimensional 
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geospatial  map  that  graphically  depicts  the  geographic  location  of  each  product  by 
use  of  a  green  square  icon.  This  icon  represents  all  products  that  were  found  at  a 
single  location.  The  three  icons  in  Figure  2.3  represent  at  least  3  products.  They  may 
also  represent  three  hundred  products  or  more. 

To  take  full  advantage  of  Project  Broadsword’s  potential,  data  visualization 
techniques  must  be  further  refined  and  exploited.  The  current  two-dimensional  map 
offers  a  data  visualization  capability  using  geospatial  coordinates  as  its  base.  As  can 
be  seen  in  Figure  2.3,  this  does  not  provide  any  meaningful  information  beyond  data 
clustering.  Since  a  single  icon  can  represent  an  unlimited  number  of  products,  each  is 
a  hypertext  link,  that  when  selected,  results  in  a  textual  output  as  seen  in  Figure  2. 1 . 
Being  forced  to  return  to  a  textual  format  immediately  after  results  are  displayed 
negates  the  true  essence  of  utilizing  data  visualization  in  the  first  place. 

2.4  Data  Visualization 

The  increasing  number  of  data  sources  and  information  repositories  coupled 
with  the  availability  of  fast  digital  network  access  through  the  Internet  and  more 
secure  global  networks  has  created  an  enormous  demand  for  querying,  accessing, 
retrieving  and  understanding  of  information.  One  approach  to  aid  in  the  organization 
of  these  tasks  involves  data  visualization.  The  effective  use  of  visualization 
addresses  the  challenge  of  discovering  and  exploiting  information  that  may  have 
otherwise  been  overlooked  or  neglected. 

Data  visualization  enhances  decision-making  through  spatial  exploitation  of 
diverse  data  products.  Technological  advances  in  computing  have  widened  the  data 
visualization  spectrum  to  a  point  where  the  greatest  challenge  is  no  longer  the 
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available  technology  but  rather  retaining  the  intimate  involvement  and  understanding 
of  the  end-user.  Traditional  text  interfaces,  which  navigate  with  a  mouse  or 
keyboard,  allow  us  to  work  on  a  computer,  interacting  with  the  system  and  not  the 
data.  Data  visualization,  on  the  other  hand,  enables  us  to  work  with  the  computer  by 
enabling  interaction  with  the  data.  This  approach  allows  the  system  to  be  an 
extension  of  our  visual  and  cognitive  system,  creating  a  seemingly  free-flow 
convergence  of  user  and  data. 

The  quest  for  the  perfect  data  visualization  implementation  is  a  moving  target 
and  presents  some  major  challenges.  The  first  task  is  to  locate  and  retrieve  the 
relevant  data.  In  our  world  of  dynamically  changing  and  diverse  heterogeneous  data 
sources,  this  in  itself  is  a  major  undertaking.  Though  not  generally  a  visualization 
function,  data  retrieval  is  a  required  step  toward  a  successful  visual  product. 

Next  is  the  challenge  of  understanding  the  data  at  a  level  that  is  appropriate 
and  comfortable  for  the  user.  Consider  the  different  levels  of  military  command,  each 
having  their  own  intelligence  requirements,  yet  accessing  the  same  data  sources. 
Identical  data  elements  may  have  a  varying  degree  of  importance  at  each  level  and 
therefore  require  a  different  priority  in  the  visual  realm.  Maintaining  a  high  level  of 
human  involvement  and  participation  in  the  data  processing  and  analysis 
requirements  through  visual  interaction  supports  this  association.  The  intimate 
involvement  and  understanding  of  data  analysis  significantly  aids  in  the  prevention  of 
erroneous  abstract  conclusions  by  ensuring  the  meaning  and  intent  of  the  data  is 
portrayed  in  context  of  the  task  at  hand,  and  not  simply  interpreted  blindly.  For 
example,  the  sudden  grounding  of  several  Minuteman  III  Payload  Transporters  (used 
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to  transport  missile  guidance  components)  may  seem  suspicious  by  an  analyst  who 
does  not  understand  Intercontinental  Ballistic  Missiles  (ICBM)  inspection 
requirements.  Effectively  communicating  the  proper  meaning  of  the  information  is 
paramount.  Portraying  a  perfectly  crafted  visualization  is  meaningless  if  the  intended 
audience  does  not  understand  or  misinterprets  it. 

A  further  challenge  of  information  visualization  is  the  available  hardware 
interfaces  and  software  display  formats.  The  goal  is  to  maximize  the  information 
content  within  our  physical  constraints  while  accommodating  and  capitalizing  on  the 
natural  capability  of  the  user.  From  a  hardware  perspective  this  is  simply  an  issue  of 
resolution  and  rendering  performance.  For  the  user,  complications  such  as  visual 
acuity  and  level  of  understanding  must  be  considered. 

2.4.1  Color 

Based  on  research  performed  by  the  Department  of  Computer  Science  at  the 
University  of  British  Columbia,  an  important  factor  to  consider  when  building  visual 
models  is  the  correct  use  of  color  (5).  Color  is  a  popular  and  effective  attribute  used 
to  map  a  dataset  into  a  visual  plane.  Previous  studies  attest  to  the  fact  that  choosing  a 
correct  color,  or  number  of  colors,  is  an  important  step  towards  a  successful 
visualization.  Research  on  the  design  of  military  systems  suggests  that  the  general 
guideline  for  computer-generated  images  is  no  more  than  five  to  seven  colors  at  a 
time  (5).  Additionally,  recent  work  at  the  IBM  Thomas  J.  Watson  Research  Center 
has  focused  on  a  rule-based  visualization  tool  that  considers  how  a  user  perceives 
visual  features  such  as  hue,  luminance,  height,  and  so  forth  (6,  7).  This  work 
emphasizes  the  understanding  of  data  visualization;  however,  it  has  not  addressed 
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how  color  aids  in  a  more  accurate  identification  of  individual  data  elements. 

The  use  of  color  is  a  powerful  medium  element  that  capitalizes  on  each 
individual’s  hard-wired  perceptual  capability.  Color  perception  is  deeply  wired  into 
the  human  psyche  and  as  a  result  our  perception  of  color  is  fast,  accurate,  automatic, 
and  effortless.  A  well-constructed  image  will  use  color  to  reduce  the  amount  of 
thought  process  required  to  understand  the  information  being  presented.  This  thought 
process,  defines  those  elements  required  for  more  complex  recognition,  such  as 
reading  text  or  attaching  meaning  to  an  object.  Thinking  is  a  relatively  slow  process 
that  requires  mental  resources  and  is  error-prone.  Misreading  a  word  or  sentence  is 
relatively  common  whereas  mistaking  a  red  apple  for  blue  is  not.  Even  those  who  are 
colorblind  easily  acknowledge  subtle  changes  in  color.  Based  on  these  founding 
principles,  color  tends  to  be  the  primary  focus  of  data  visualizations. 

In  three-dimensional  space,  color  is  an  effective  tool  for  the  control  of  spatial 
perception  (8).  In  a  natural  scene,  mountains  off  in  the  distance  usually  appear  blue- 
violet  and  indistinct.  Known  as  aerial  perspective,  this  visual  effect  is  produced  by 
white  sunlight  hitting  the  atmosphere  and  refracting  on  its  shorter  wavelength  (8). 
Since  the  eye  automatically  interprets  blue-violet  and  loss  of  detail  as  a  sign  of 
distance,  it  can  be  creatively  used  to  add  a  feeling  of  depth  to  a  flat  display.  To 
emphasize  closeness,  warm  colors  such  as  red,  yellow,  and  orange  are  used. 

2.4.2  Vision 

Effective  graphics  development  requires  a  general  understanding  of  vision  and 
the  distinction  between  “attentive”  and  “preattentive”  processing  (8).  At  the  instance 
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a  scene  is  first  viewed,  there  is  more  information  presented  than  can  be  cognitively 
processed.  To  overcome  this,  we  instinctively  decompose  the  image  into  more 
manageable  pieces  and  then  examine  each  piece  individually.  This  divide  and 
conquer  strategy  is  called  attentive  processing  and  is  accomplished  by  moving  the 
eyes  such  that  the  current  area  of  interest  forms  an  image  on  the  fovea,  the  highest 
acuity  area  of  the  retina.  Attentive  processing  can  be  thought  of  as  a  spotlight,  which 
the  viewer  moves  across  the  scene.  Once  formed,  the  highlighted  segments  are 
processed  for  further  meaning,  such  as  understanding  an  object’s  representation,  and 
are  no  longer  simply  patches  of  color.  In  this  attentive  processing  stage,  the  viewer  is 
still  dimly  aware  of  the  sensations  outside  of  the  spotlighted  area  but  has  no  idea  of 
their  meaning.  While  focusing  on  the  local  area  the  viewer  still  maintains  a  sense  of 
awareness  of  the  rest  of  the  world. 

To  effectively  perform  attentive  processing  the  user  must  effectively  break  the 
image  into  useful  segments  and  then  decide  which  area  to  examine  first.  Preattentive 
processing  provides  this  function  by  relying  on  raw  sensory  information,  such  as 
color,  which  has  no  immediate  inherent  meaning.  As  the  user  organizes  and  links 
different  colors  encountered,  natural  image  segmentation  takes  place.  This 
instinctiveness  allows  a  similar  color  in  geographically  separate  areas  to  reinforce 
commonality.  Figure  2.4  illustrates  how  a  designer  can  use  color  to  push  a  user’s 
view  in  a  desired  direction  (8).  Notice  how  the  red  squares  on  the  right  immediately 
draw  attention  from  the  dark  blue  squares  and  involuntarily  force  you  to  link  them 
into  a  single  line.  The  lighter  blue  squares  on  the  left  differ  only  in  brightness  and 
therefore  the  effect  is  not  as  strong.  Both  of  these  examples  demonstrate  how  a  clever 
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Figure  2.4:  Preattentive  Processing  (Color  Plate  2) 


designer  can  use  preattentive  processing  to  draw  a  user’s  eye  to  a  desired  location, 
and  further,  control  to  some  extent,  the  movement  of  the  eye  around  an  image. 

The  importance  of  optimizing  preattentive  processing  and  minimizing 
thinking  is  widely  acknowledged  in  visualization  literature.  Cleaveland  noted  that  in 
“elementary  graphical-perception  tasks”  the  preattentive  perception  of  basic  graphical 
elements  underlies  data  visualization  (9).  Abarbanel  suggests  that  visualization  can 
be  defined  as  the  substitution  of  “preconscious  visual  competencies”  for  “conscious 
thinking”(10).  Woods  indicated  that  “If  the  mental  activities  required  to  locate  base 
data  units  are  not  automatic,  but  require  mental  effort  or  capacity,  they  may  disrupt  or 
divert. .  .the  main  line  of  reasoning”  (11). 

2.5  Conclusion 

Intelligence  plays  a  vital  role  in  the  decision  making  process  both  in  and  out 
of  a  military  conflict.  Reaching  these  decisions  is  no  small  task.  Navigating  the  sea 
of  information  through  our  vast  data  repositories  is  a  large  undertaking.  Confronting 
this  intelligence  problem  is  Project  Broadsword.  While  making  great  strides  in  the 
area  of  data  management.  Project  Broadsword  is  also  investigating  more  advanced 
data  exploitation  methods  through  visual  cues.  The  field  of  data  visualization,  though 
constrained  by  technology  and  human  factors,  is  an  expanse  of  possibilities  waiting  to 
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be  explored.  Exploration  of  the  visual  realm  leads  immediately  to  the  use  of  color. 
Association  through  color  is  an  inherent  human  trait  that  is  very  fast,  efficient,  and 
effortless.  This  research  embraces  that  fact  and  exploits  it. 


19 


3.  Methodology 


3.1  Data  Exploration 

This  research  provides  an  effective  representative  environment  for 
exploitation  of  stored  military  intelligence.  The  philosophy  of  design  is  to  provide 
the  intelligence  analyst  with  enhanced  visual  cues  and  the  spatial  freedom  to  explore, 
analyze,  and  evaluate  products  from  a  chosen  geographical  area  of  interest  in  three- 
dimensional  space.  The  Project  Broadsword  Test  and  Evaluation  team  conducted 
intelligence  product  exploitation  tests  with  and  without  the  use  of  data  visualization 
techniques.  Their  subjective  comparisons  between  data  visualization  and  a  text-only 
approach  lay  the  groundwork  for  the  findings  in  Chapter  6  and  the  conclusions  and 
recommendations  in  Chapter  7. 

3.2  Software  Development  Process 

The  nature  of  the  Project  Broadsword  data  visualization  problem  lends  itself 
nicely  to  the  construction  of  a  partial  implementation,  or  prototyping.  Within 
prototyping  there  are  two  schools  of  thought,  the  throwaway  approach  and  the 
evolutionary  approach  (12).  The  throwaway  approach  is  concerned  with  the 
construction  of  software  in  order  to  learn  more  about  the  problem  or  its  solution. 
Once  this  is  understood,  the  prototype  is  completely  discarded.  The  evolutionary 
approach  also  examines  the  problem  or  its  solution;  however,  once  the  requisite 
knowledge  has  been  gained,  the  prototype  is  readapted  and  used  again.  This  process 
repeats  itself  until  the  prototype  eventually  becomes  the  actual  system. 

The  intelligence  community  is  beginning  to  explore  how  the  use  of  data 
visualization  can  satisfy  its  users’  needs.  Responding  to  this  trend.  Project 
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Broadsword  is  emphasizing  the  “look  and  feel”  of  a  final  data  visualization  design, 
not  its  underlying  implementation.  This  research  provides  the  means  by  which  an 
analyst  can  explore  and  adapt  to  new  data  visualization  ideas  and  concepts.  For  this 
reason,  I  have  chosen  throwaway  prototyping  as  the  software  development  process. 

3.3  Visualization  Considerations 

Applying  methods  of  good  data  visualization  techniques  can  be  difficult. 
Although  a  significant  amount  of  background  work  goes  into  producing  “good”  data 
visualizations,  a  large  part  of  this  thesis  relies  on  visual  experimentation  and  the 
associated  human  response.  Regardless  of  technical  orientation,  each  Project 
Broadsword  tester’s  feedback  is  partially  subjective  and  based  on  their  personal  tastes 
and  prior  data  visualization  experience.  Final  analysis  of  this  research  and  the 
associated  conclusions  must  take  this  subjectivity  into  account.  This  research  uses 
technically  sound  approaches  to  explore  appealing  graphical  representations. 
Technically  perfect  data  visualizations  will  not  be  utilized  if  they  do  not  appeal  to  the 
user. 

As  stated  in  Section  2.4.1,  color  is  a  major  factor  in  the  success  or  failure  of 
data  visualizations.  For  this  reason,  color  is  heavily  examined  throughout  this 
research.  Evaluating  the  effects  of  different  colors  at  varying  levels  of  resolution  and 
intensity  account  for  a  significant  amount  of  effort. 

Transparency  is  a  unique  graphics  capability  that,  when  used  effectively, 
allows  for  the  interior  of  a  three-dimensional  object  to  be  used  as  additional  data 
presentation  space.  This  research  explores  the  additional  capabilities  provided  by  this 
method  of  object  creation. 
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3.4  Data  Navigation 

Intelligence  products  are  constructed  with  more  data  elements  than  can  be 
comfortably  mapped  to  the  visual  realm.  Therefore,  a  successful  visualization  is 
dependent  on  the  environment  and  its  ability  to  handle  this  overload.  When  viewed 
from  this  perspective,  data  visualization  takes  a  back  seat  to  navigation  and  is  handled 
accordingly  throughout  this  research. 

3.5  Scenario  Generation 

To  validate  the  results  of  this  research,  a  scenario  was  constructed  which 
simulates  real-world  intelligence  product  exploitation.  This  scenario,  seen  in  Figure 
3.1,  exercises  the  current  Project  Broadsword  textual  and  two-dimensional  product 
retrieval  functions.  The  scenario  is  then  exercised  using  the  newly  developed  three- 
dimensional  data  visualization  methods. 

For  each  experiment,  testers  are  provided  with  the  result  formats  and  tasks  as 
outlined  in  Figure  3.1.  Providing  the  results  eliminates  the  need  for  testers  to  perform 
the  actual  searches  and  ensures  each  tester  has  an  identical  display  at  the  start  of  each 
test.  Following  each  test,  feedback  is  generated  through  tester  interviews.  Each  tester 
subjectively  rates  the  experience  for  ease  of  use,  suitability  to  task  and  perceived 
performance.  Findings  and  conclusions  based  on  a  comparison  of  test  results  are 
found  in  Chapters  6  and  7. 

The  scenario  in  figure  3.1,  used  for  testing  purposes,  is  constructed  of 
intelligence  products  from  three  intelligence  sources  (the  Military  Intelligence 
Database  (MIDB),  the  Image  Product  Library/Image  Product  Archive  (IPL/IPA)  and 
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the  Automated  Message  Handling  System  (AMHS))  and  contains  all  four  product 
types;  i.e.  imagery,  video,  text  and  audio. 


Broadsword  Geospatially  Oriented  Navigational  Environment 

Experiment  Scenario 


Instructions:  Given  a  set  of  one  hundred  and  seventeen 

intelligence  products  and  three  result  formats,  complete  the 
three  tasks  in  any  order.  Utilize  the  accompanying  color 
identification  map  for  product-to-glyph  color  associations. 

Result  Formats:  1.  Text-only 

2.  Two-dimensional  map  with  icons 

3.  Three-dimensional  map  with  glyphs 


Task  #1:  Locate  the  total  number  of  textual  products 

available . 

Task  #2:  Retrieve  all  imagery  products  that  have  IPL  as 

a  data  source. 

Task  #3:  Identify  all  F-15E  aircraft  imagery 

products. 


Intelligence  Product -to-Glyph  Color  Identification  Mapping 


Color 

Product  Type 

Green 

Imagery 

Purple 

Textual 

Red 

Video 

Yellow 

Audio 

Gray 

Multiple  Products 

Figure  3.1:  Experiment  Scenario 
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4.  Development  Environment 


4.1  Environment 

To  confront  the  growing  stream  of  intelligence  data,  as  stated  in  Section  1.3, 
while  meeting  the  challenges  of  data  management  as  outlined  in  Section  1.4,  this 
research  effort  created  an  environment  to  explore  proof-of-concept  data  visualization 
capabilities  tailored  to  the  Project  Broadsword  intelligence  data  retrieval  and 
exploitation  system.  This  environment,  the  Broadsword  Geospatially  Oriented 
Navigational  Environment  (BGONE),  was  constructed  in  accordance  with  the  Project 
Broadsword  System  Security  Requirements  and  Analysis  document.  This  document 
provides  the  vehicle  for  listing,  and  establishing  compliance  with,  the  minimum 
technical  and  nontechnical  requirements  for  the  Project  Broadsword  System  in 
processing  U.S.  intelligence  in  the  System  High  mode  (classified  mode)  of  operation. 
Additionally,  it  provides  an  analysis  of  Project  Broadsword’s  safeguards  intended  to 
satisfy  system  security  requirements  where  applicable. 

Due  to  the  sensitive  nature  of  Project  Broadsword’s  mission,  BGONE  is 
constrained  to  coding  languages  and  techniques  authorized  by  the  Project  Broadsword 
program  office.  These  constraints  constricted  BGONE’ s  evolution  by  significantly 
reducing  the  number  of  possible  development  paths. 

4.2  Target  Environment 

Although  BGONE  is  a  stand-alone  environment,  its  implementation  mirrors 
the  functional  design  of  Project  Broadsword.  A  review  of  available  documentation 
and  operational  parameters  revealed  that  Project  Broadsword  is  deeply  committed  to 
the  cross  platform  belief.  It  utilizes  only  those  processes  and  techniques  to  interact 
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with  the  system  from  virtually  any  platform.  Additionally,  the  system  is  carefully 
tailored  to  include  only  those  programming  languages  that  support  seamless  operation 
through  an  Internet  Protocol  network.  These  languages  include  the  Hypertext 
Markup  Language  (HTML),  JavaScript,  the  Common  Gateway  Interface  (CGI)  and 
the  Virtual  Reality  Modeling  Language  (VRML).  Through  strict  adherence  to  these 
developmental  policies,  users  are  able  to  perform  their  duties  by  simply  utilizing  a 
standard  World  Wide  Web  (WWW)  browser  such  as  Internet  Explorer  or  Netscape 
Navigator.  To  ensure  accurate  results,  BGONE  conforms  to  this  operating 
environment  paradigm. 

4.3  Project  Broadsword  Code  Base 

Project  Broadsword  is  currently  undergoing  a  major  code  rewrite.  Of  key 
importance  to  this  research  is  the  map  server  and  its  associated  data  handling 
functions.  As  these  are  being  re-coded  from  the  ground  up,  their  completion  times 
exceed  that  allotted  for  this  study.  To  support  this  research  without  having  a  fully 
functional  Project  Broadsword  map  server,  BGONE  was  created  as  a  stand-alone 
entity.  The  new  map  server  relies  heavily  on  VRML  and  JavaScript  code;  therefore 
these  languages  make  up  BGONE’ s  core  code  base. 

4.4  Model  Approach 

BGONE  provides  exploration  of  intelligence  data  handling  concepts  through 
the  use  and  manipulation  of  three-dimensional  VRML  models.  BGONE’ s  primary 
focus  is  the  representation  of  the  data,  and  not  the  terrain  over  which  it  is  displayed. 
Therefore,  three-dimensional  maps  are  externally  generated  and  imported  as  needed. 
The  Project  Broadsword  development  team  provided  map  models  as  needed. 
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4.5  Evaluation  of  Hardware  Platforms 


Multiple  hardware  platforms  are  available  within  the  Project  Broadsword 
development  lab.  These  platforms  include  SUN,  Silicon  Graphics,  and  x86  based 
machines  of  varying  speeds  and  capabilities.  Based  on  suitability  to  task,  ease  of  use, 
and  support  for  VRML,  the  x86  architecture  was  chosen  as  the  development  platform. 
Although  SUN  systems  are  prevalent  in  the  lab,  UNIX  based  platforms  do  not  yet 
fully  support  the  VRML  language. 

4.6  Software  Development  Languages 

BGONE  was  developed  using  JavaScript,  HTML,  and  VRML.  These 
languages  were  pre-determined  by  the  Project  Broadsword  project  office  so  no 
evaluation  of  alternate  tools  was  performed.  VRML  was  chosen  as  the  only  available 
three-dimensional  graphics  language  that  adheres  to  Project  Broadsword’s  strict 
security,  integrity  and  IP  requirements.  VRML  provides  a  robust  graphics-rendering 
library  but  lacks  functionality  in  the  area  of  data  navigation.  JavaScript  and  HTML 
fill  the  void  by  providing  effective  data  selection  and  interaction. 

BGONE  is  primarily  based  on  the  1997  International  Organization  for 
Standardization  (ISO)/Intemational  Electrotechnical  Commission  (IEC)  document 
14772,  Virtual  Reality  Modeling  Language  (13).  All  VRML  code  is  constructed 
using  this  document  as  a  guide.  Additionally,  the  JavaScript  and  HTML  support  code 
are  integrated  using  established  industry  practices. 

4.7  Automated  Tools 

To  facilitate  the  development  of  BGONE,  I  employed  the  use  of  Spazz3D  by 
Virtock  Technologies,  a  powerful  VRML  object  generation  tool.  Spazz3D  provides 
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key  functions  which  include: 

•  A  GUI  based  interface  to  accommodate  simple  geometry  creation  by  a 
visual  drag-and-drop  method 

•  Software  wizards  that  aid  complex  tasks  such  as  creating  animations  and 
moving  cameras. 

•  OpenGL  support  for  enhanced  graphics  rendering 

•  Easily  implemented  directional  lights,  point  lights,  and  spot  lights 
While  Spazz3D  provided  the  bulk  of  objects  used  in  BGONE,  assembling  the 

environment  and  employing  data  navigation  methods  required  coding  by  hand.  This 
was  accomplished  with  PC  Editor,  a  simple  multi-windowed  ASCII  text  editor. 

4.8  Graphics  Rendering 

BGONE  graphical  rendering  is  accomplished  using  OpenGL.  Developed  by 
Silicon  Graphics  Incorporated  in  1992,  OpenGL  is  the  most  widely  adopted  graphics 
standard  in  use  today  and  is  supported  on  virtually  all  platforms.  This  fact  coupled 
with  its  ability  to  produce  visually  consistent  displays,  regardless  of  operating  system 
or  windowing  system,  maintains  Project  Broadswords’  goal  of  universal  functionality 
and  presentation  across  heterogeneous  platforms.  When  OpenGL  compliant 
hardware  is  unavailable  or  not  configured,  software  emulation  is  seamlessly  enabled. 

4.9  External  Support 

BGONE,  developed  with  interpretive  based  languages,  requires  a  JavaScript 
enabled  WWW  browser  with  an  associative  VRML  plug-in  to  function  properly. 
Since  no  web  browser  supports  VRML  code  directly,  a  helper  program  or  plug-in  is 
required.  Although  the  code  in  BGONE  is  based  on  VRML97,  JavaScript  1.2,  and 
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HTML  3.2  standards,  each  browser  supports  a  particular  standard  with  varying 
degrees  of  success.  Due  to  their  lack  of  compliance  and  poor  documentation  on  the 
part  of  VRML,  JavaScript  and  HTML;  the  success  or  failure  of  a  specific  design  can 
only  be  validated  following  completion.  The  testing  of  the  various  combinations  of 
web  browsers  and  plugins  played  a  crucial  role  in  BGONE  development  and  is 
explained  in  section  4.10. 

4.10  Commercial  Off  The  Shelf  Products 

Project  Broadsword  prides  itself  in  leveraging  off  of  existing  technologies. 

Due  to  its  web  design,  it  relies  on  commercially  based  WWW  browsers  and  VRML 
plugins.  There  are  three  major  players  with  respect  to  browsers:  Internet  Explorer, 
Netscape,  and  Opera.  Internet  Explorer  is  a  Microsoft  standard  browser  and  is 
available  for  the  x86  Windows  and  Sun  Solaris  based  architectures.  Netscape  caters 
to  virtually  every  architecture.  Opera  is  a  x86-based  browser  that  is  under 
development.  Of  these  three,  Netscape  is  the  primary  browser  used  by  the  Project 
Broadsword  development  team  as  it  has  the  most  support.  Based  on  this  fact, 
Netscape  was  also  chosen  as  the  web  viewer  for  BGONE. 

As  inferred  in  section  4.9,  Netscape  does  not  support  a  VRML  environment 
directly.  It  requires  an  external  plugin  to  perform  this  function.  Three  VRML 
plugins  were  considered  for  use:  Cosmo,  Blaxxun  Contact  3D  and  Cortona.  Cosmo 
has  been  a  leader  in  VRML  technology  in  the  past  but  development  has  been  halted 
and  it  is  no  longer  supported.  Blaxxun  Contact  3D  and  Cortona  are  currently  the  only 
two  actively  supported  VRML  plugins.  Both  take  advantage  of  OpenGL  accelerated 
hardware  while  providing  adequate  navigational  functions.  The  only  true 
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differentiation  is  the  amount  of  real  estate  used  by  the  on-screen  VRML  controls. 
Blaxxun  Contact  3D  excels  in  this  area;  utilizing  a  relatively  small  footprint  and  thus 
was  chosen  as  the  BGONE  companion  to  Netscape. 
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5.  Design  and  Implementation 


5.1  Environment 

The  Broadsword  Geospatially  Oriented  Navigational  Environment  (BGONE) 
is  a  stand-alone  software  environment  that  provides  an  effective  representative 
atmosphere  for  exploitation  of  stored  military  intelligence.  Modeled  for  Project 
Broadsword,  BGONE  is  a  stand-alone  environment  that  provides  an  effective 
representative  atmosphere  for  exploitation  of  stored  military  intelligence.  BGONE 
design  and  implementation  is  based  on  design  concepts  of  the  new  Project 
Broadsword  3.x  map  server.  It  also  complies  with  policies  and  procedures  congruent 
with  mandated  security  requirements  outlined  in  Section  4.2. 

BGONE  provides  a  one-to-one  mapping  of  intelligence  data  product  metadata 
elements,  such  as  product  type,  to  a  specific  visual  attribute  like  color.  Mapping  is 
provided  by  pre-selected  user  input  and  is  a  function  of  the  environment  architecture. 
Project  Broadsword  prides  itself  in  offering  the  user  a  plethora  of  configurable 
options  when  using  the  system.  Likewise,  the  integration  of  concepts  and  techniques 
developed  in  BGONE  are  intended  to  be  user  configurable  via  the  Project 
Broadsword  user  preference  interface.  Hence,  each  individual  user  can  select  which 
visual  attribute  maps  to  which  data  element  to  satisfy  their  specific  needs. 

The  most  important  contribution  of  BGONE  is  the  ability  to  map  intelligence 
data,  not  decide  what  the  ideal  mapping  should  be.  For  the  purposes  of  this  research, 
Table  5.1  depicts  the  mapping  of  color  to  product  type,  i.e.  imagery,  video,  text  or 
sound.  Additionally,  the  use  of  the  color  gray  designates  that  multiple  products  are 
present  but  not  shown.  Use  of  this  particular  mapping  along  with  others  will  be 
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further  discussed  throughout  this  chapter. 


Table  5.1:  Intelligence  Data  Mapping 


Product  Type 

Visual  Attribute  -  Color 

Imagery 

Green 

Textual 

Purple 

Video 

Red 

Audio 

Yellow 

5.2  Three-dimensional  Visualization 

User  feedback  obtained  through  informal  interviews  revealed  a  need  for  an 
environment  where  intelligence  products  could  be  exploited  through  visual  means. 
Leveraging  off  of  the  old  adage  that  “a  picture  is  worth  a  thousand  words”,  user’s 
expressed  the  desire  for  a  mechanism  that  provided  increased  understanding  of  a 
given  set  of  intelligence  products  within  a  narrow  window  of  time.  Specifically 
implicated  was  the  additional  possibilities  generated  by  the  use  of  the  third 
dimension.  Adding  this  additional  dimension  significantly  increases  the  amount  of 
usable  virtual  real  estate  for  data  presentation. 

Responding  to  user  feedback,  this  research  did  not  pursue  two-dimensional 
data  visualization.  Instead,  BGONE  moves  directly  from  a  textual  to  a  three- 
dimensional  representation.  This  transition  poses  technical  obstacles  that  are  a  non¬ 
issue  in  two  dimensions.  These  obstacles  include  occlusion  as  objects  are  placed 
directly  in  front  of  each  other,  user  perspective  at  a  given  instance  in  time,  and  data 
navigation. 

5.2.1  Three-dimensional  Objects 

Managing  representative  objects  in  data  visualization  begins  with  the  creation 
of  the  objects  themselves.  To  accommodate  BGONE  development  a  glyph-based 
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visualization  approach  was  employed.  A  glyph  is  an  icon  or  graphical  object  that  is 
affected  by  its  input  data  and  certain  properties  such  as  x,  y,  z  locations  in  three- 
dimensional  space,  color,  opacity,  orientation,  shape  and  size. 

Glyphs  differ  from  standard  graphical  objects  because  they  provide  a  direct 
one-to-one  mapping  between  a  selected  data  element  and  a  glyph  attribute.  For 
example,  an  intelligence  product  type  can  be  directly  mapped  to  the  color  attribute. 
Using  the  mapping  of  table  5.1,  figure  5.1  symbolizes  four  imagery  products.  Since 
figure  5.1  displays  four  unique  shapes,  a  second  data  element,  such  as  the  branch  of 
military  service  that  produced  the  imagery,  could  be  mapped  to  the  shape  attribute. 
This  second  mapping  would  increase  the  usefulness  of  the  data  visualization  by 


Figure  5.1:  Symbolic  Glyphs  (Color  Plate  3) 
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helping  the  analyst  visually  segregate  the  search  space  by  product  and  image  creation 
source.  As  more  ‘intelligence  data  element  to  glyph  attribute’  mappings  are 
performed,  the  given  search  space  is  further  populated  with  more  graphically 
represented  product  details.  Each  added  representation  increases  the  analysts’  ability 
to  quickly  select  specific  products  of  interests  without  immediate  textual  intervention. 

For  BGONE,  initial  prototype  glyphs  were  created  using  Spazz-3D,  which  is 
described  in  section  4.10  Automated  Tools.  Using  Spazz-3D  it  was  possible  to 
quickly  create  glyphs,  such  as  those  seen  in  figure  5.1,  for  immediate  examination. 
5.2.2  Multilevel  Glyphs 

Users  specifically  requested  three-dimensional  data  visualization.  Their 
request  was  motivated  by  the  knowledge  that  using  the  third  dimension  allows  for  the 
use  of  depth,  which  provides  more  usable  space  for  data  presentation.  To  ensure 
maximum  screen  space  usage,  glyphs  were  created  at  two  levels  of  data  resolution; 
level  one  and  level  two.  A  level  one  glyph,  often  referred  to  as  the  outer  glyph,  is  the 
primary  glyph  and  provides  identification  of  individual  products  such  as  imagery. 
Each  level  one  glyph  represents  one  intelligence  product.  A  level  two  glyph,  often 
referred  to  as  the  inner  glyph,  is  the  secondary  glyph  and  provides  specific 
information  about  that  particular  product.  Figure  5.2  shows  representation  of  two 
products.  Based  on  this  example,  the  green  level  one  glyphs  symbolize  two  imagery 
products.  The  level  two  glyph,  which  can  be  seen  inside  the  level  one  glyph,  can  now 
further  the  analysts’  understanding  of  the  product  through  a  one-to-one  matching  of 
product  element  to  glyph  attribute.  Here  a  cube  may  represent  ground-based  imagery 
while  the  cone  may  represent  air-based  imagery.  Additionally,  the  color  of  the  inner 
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glyph  offers  another  mapping  level  such  as  product  origin.  As  shown,  this  glyph  in  a 
glyph  visual  approach  helps  maximize  the  useable  space  taken  up  by  each  product 
representation.  Although  this  does  not  effect  the  number  of  products  displayed,  it 
does  increase  the  amount  of  information  portrayed. 


Figure  5.2:  Multilevel  Glyphs  (Color  Plate  4) 


5.2.2. 1  Level  One  Glyph 

To  allow  the  analyst  to  identify  each  intelligence  product,  a  level  one  glyph 
represents  one  product  in  its  entirety.  The  first  area  of  interest  dealt  with  the  shape 
used  when  displaying  this  glyph.  The  use  of  shape  as  a  data-mapping  attribute 
required  careful  examination.  Initially,  each  of  the  four  product  types  was  mapped  to 
one  of  the  geometric  shapes  shown  in  figure  5.1.  Preliminary  testing  indicated  that 
the  vertical  stacking  of  these  glyphs  in  relatively  small  concentrations  (20  products  or 
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less)  resulted  in  an  acceptable  representation.  However,  when  the  number  of  products 
displayed  increased,  the  virtual  landscape  began  to  take  on  a  cluttered  appearance. 
After  developing  several  models  and  consulting  with  the  Project  Broadsword  program 
office,  it  was  decided  to  maintain  a  uniform  shape  for  all  outer  glyphs. 

Four  common  geometric  shapes  were  candidates  for  the  level  one  glyph:  the 
cube,  the  sphere,  the  cone  and  the  cylinder.  Each  was  evaluated  for  its  appearance 
after  being  vertically  stacked  and  placed  side-by-side.  Based  on  the  subjective 
observation  that  they  do  not  stack  well  and  wasted  the  most  space  per  object,  the  cone 
and  sphere  were  eliminated  as  contenders.  As  can  be  seen  in  figures  5.3  and  5.4,  the 
two  remaining  choices  offer  a  uniform  eye  appealing  presentation.  The  final  decision 


Figure  5.3:  Cylinder  Shaped  Level  One  Glyphs  (Color  Plate  5) 
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to  use  a  cube-based  glyph  was  made  to  reduce  the  number  of  polygons  required  to 
generate  each  glyph. 


Figure  5.4:  Cube  Shaped  Level  One  Glyphs  (Color  Plate  6) 


With  the  cube  selected  as  the  level  one  glyph,  attention  was  turned  to  the  use 
of  color.  As  explained  in  section  2.4.1,  color  is  a  very  effective  data  visualization 
tool.  For  the  purpose  of  this  research,  the  color  of  the  outer  glyph  represents  a 
specific  product  type.  Table  5.1  provides  the  color  mapping  used. 

Exploiting  the  inner  glyph  only  succeeds  if  the  outer  glyph  provides  visual 
access  to  its  interior.  One  approach  is  to  use  transparency.  Transparency  provides 
interior  access  while  maintaining  the  outer  glyphs  shape  and  color  integrity.  This 
technique  also  allows  a  user  to  see  other  products  in  the  distance  without  changing 
perspective.  Figures  5.5  and  5.6  examine  varying  levels  of  transparency  and  the 
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Figure  5.5:  Transparency  as  Designed  (Color  Plate  7) 


ability  to  distinguish  the  inner  glyph  while  maintaining  the  outer  glyph.  Figure  5.5 
displays  transparency  as  viewed  with  the  Spazz-3D,  a  VRML  object  generation  tool. 
Using  Netscape  and  its  associated  VRML  plugin  Blaxxun  Contact  3D,  as  required  by 
Project  Broadsword,  resulted  in  figure  5.6.  Examining  these  two  figures  highlights 
two  major  inconsistencies.  First,  figure  5.5  demonstrates  six  levels  of  transparency  as 
designed.  Notice  that  at  each  level,  both  the  inner  and  outer  glyph  is  discernable. 
Figure  5.6  shows  the  same  six  glyphs,  as  viewed  in  an  operational  environment.  It  is 
clearly  evident  that  as  the  level  of  transparency  increases  the  visibility  of  the  outer 
glyph  diminishes  until  it  is  no  longer  discernable. 

Another  adverse  inconsistency  is  the  textural  appearance  of  the  outer  glyph. 
The  original  design,  figure  5.5,  was  intended  to  provide  a  smooth  fog  like  effect,  not  a 
coarse  texture.  The  coarseness  displayed  in  the  top  row  of  figure  5.6  is  easily 
mistaken  as  intended  texture.  Although  mapping  intelligence  data  to  texture  may  lead 
to  more  data  represented,  this  was  not  the  intent  and  it  visually  detracts  from  the  inner 
glyph.  This  detraction  reduces  the  inner  glyphs  ability  to  effectively  provide  data 
visualization  mappings  by  making  it  difficult  to  see,  thus  reducing  its  utility.  Being  in 
three-dimensional  space,  the  coarseness  also  reduces  the  ability  to  see  objects  directly 
behind  the  glyph  being  viewed. 

The  diminished  visual  quality,  as  is  evident  by  the  lack  of  definition  and 
coarseness  in  figure  5.6,  severely  impacted  the  decision  to  pursue  transparency.  The 
dismal  results  of  transparency  further  emphasized  the  lack  of  VRML  compatibility 
from  one  VRML  interpreter  to  another. 
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A  second,  more  successful  approach  to  outer  glyph  creation  is  shown  in  figure 
5.7.  The  outer  glyph  is  constructed  of  twelve  smaller  three-dimensional  boxes  that 


Figure  5.7:  Redesigned  Level  One  Glyph  (Color  Plate  9) 


are  arranged  to  form  the  original  cube.  This  layout  maintains  a  uniform  color  mapped 
cube  while  providing  easy  visual  access  to  its  interior.  An  added  feature  of  this 
scheme  is  that  the  transparent  interior  does  not  alter  the  appearance  or  color  of  the 
inner  glyph. 

5. 2.2.2  Level  Two  Glyph 

Level  two  glyph  development  initially  progressed  as  a  mirror  of  level  one. 
Results  of  this  are  shown  in  figure  5.8.  As  shown  here  with  the  outer  green  cube 
symbolizing  an  image  product,  the  inner  glyph  attributes  of  shape  and  color  are 
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Figure  5.8:  Abstract  Level  Two  Glyph  (Color  Plate  10) 


available  for  mapping  two  additional  product  elements.  Collaborative  meetings  with 
the  Project  Broadsword  program  office  revealed  a  desire  for  a  more  gradual 
introduction  of  data  visualization  with  a  smaller  learning  curve.  Figure  5.8  requires 
the  user  to  have  a  previous  knowledge  of  the  inner  glyph  mappings.  A  more  intuitive 
approach  is  to  integrate  a  VRML  model  of  the  product  type  being  represented.  This 
is  demonstrated  in  figure  5.9  with  an  imagery  product  of  an  aircraft. 

5.2.3  Level  Of  Detail 

To  minimize  visual  clutter  when  viewing  a  large  number  of  products  from  a 
distance,  a  Level  Of  Detail  (LOD)  scheme  was  employed  in  two  stages.  First,  low 
LOD  models  were  created  to  give  analysts  a  visual  understanding  of  the  level  one 
glyph  without  requiring  knowledge  of  the  level  two  glyph.  These  models,  as  seen  in 
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Figure  5.9:  Model  Based  Level  Two  Glyph  (Color  Plate  11) 

figure  5.10,  render  only  the  outer  glyph.  Masking  the  inner  glyph  permits  the  outer 
glyph  to  be  rendered  as  a  solid  cube.  A  solid  cube  provides  more  surface  area  for  the 
color  attribute,  as  compared  to  figure  5.7,  allowing  the  viewer  to  more  easily  identify 


Figure  5.10:  Low  Level  Of  Detail  Glyphs  (Color  Plate  12) 
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a  specific  glyph  from  a  distance.  When  viewing  data  in  a  three-dimensional 
landscape,  the  ability  of  the  user  to  see  all  rendered  elements  is  crucial. 

Users  select  a  key  product  element  to  be  mapped  to  the  color  of  a  level  one 
glyph.  Their  selection  allows  them  to  quickly  locate  products  of  interest  based  on  the 
task  at  hand.  For  example,  an  analyst  may  be  directed  to  analyze  all  available 
imagery  over  a  specific  geographic  area.  If  the  inquiry  returns  a  mix  of  products,  low 
LOD  models  with  a  color  attribute  of  green  will  be  readily  evident.  As  seen  in  figure 
5.1 1,  low  LOD  models  used  with  a  VRML  landscape  map  provide  immediate 
feedback  without  the  distraction  of  individual  level  two  models. 


Figure  5.11:  Low  Level  Of  Detail  Glyphs  on  a  VRML  Landscape  Map  (Color  Plate  13) 


Low  LOD  models  minimize  visual  clutter  by  using  proximity  sensors.  As  an 
analyst  moves  closer  to  glyphs  of  interest,  the  models  change  to  that  seen  in  figure 
5.12,  revealing  additional  information  about  the  products.  When  the  analyst  is 
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finished  in  one  location  and  moves  towards  another,  the  first  glyphs  revert  to  their 


original  form  as  shown  in  figure  5.10. 


Figure  5.12:  Proximity  Sensor  Activation  Rendering  (Color  Plate  14) 

Level  Of  Detail  is  also  concerned  with  the  number  of  products  displayed  at  a 
given  time.  Even  with  three  dimensions,  display  space  is  a  limited  commodity  and 
must  be  managed  accordingly.  Project  Broadsword  testers  frequently  perform 
searches  that  result  in  over  eight  hundred  products.  To  accommodate  such  a  large 
number,  the  gray  glyph  at  the  bottom  of  Figure  5.12  was  introduced  to  signify  that 
multiple  products  are  present  but  not  visible. 

For  this  research.  Figure  5.10  or  Figure  5.12  provides  an  understanding  of 
seven  products;  textual,  imagery,  and  five  that  require  additional  actions  to  explore. 
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A  gray  glyph  is  explored  by  selecting  it  with  the  mouse  and  clicking  a  mouse  button. 
A  second  web  browser  window  is  then  generated,  as  seen  in  figure  5.13,  which 
displays  the  products  found  at  that  location. 


Figure  5.13:  Products  Represented  by  Gray  Glyph  (Color  Plate  15) 


5.3  Navigation 

When  dealing  with  a  two-dimensional  plane,  navigation  is  primarily 
concerned  with  how  the  eye  traverses  the  scene.  This  is  controlled  through  effective 
use  and  placement  of  lines,  shapes  and  color.  Entering  the  third  dimension  introduces 
an  expanded  domain  in  which  the  user  must  physically  navigate  the  data. 

Individual  intelligence  products  provide  a  large  amount  of  data  and  therefore  a 
one-to-one  matching  of  each  data  element  to  a  visual  cue  is  unrealistic.  Key  product 
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metadata  elements  are  therefore  used  for  visual  rendering  leaving  the  remainder  for 
textual  displays.  This  allows  for  quick  visual  selection  of  a  desired  product.  This 
separation  in  data  and  metadata  necessitates  the  need  for  two  types  of  navigation: 
visual  environment  navigation  and  data  navigation. 

5.3.1  Graphics  Navigation 

Visual  environment  navigation  is  provided  through  Blaxxun  3D  as  discussed 
in  section  4.10.  Table  5.2  provides  a  cursory  navigation  overview.  Particular 
keyboard  and  mouse  actions  for  each  of  the  stated  movements  can  be  found  in  the 
Blaxxun  3D  help  menu. 


Table  5.2:  VRML  Navigation 


Walk 

Allows  the  user  to  move  through  the  scene 
in  an  X  and  Z  direction 

Slide 

Allows  the  user  to  move  through  the  scene 
in  an  X  and  Y  direction 

Examine 

Allows  an  entire  scene  to  be  rotated  around 
its  own  X  axis 

Rotate 

Allows  an  entire  scene  to  be  rotated  around 
its  own  X  and  Z  axis 

Fly 

Allows  the  user  to  move  through  the  scene 
in  an  X,  Y,  and  Z  direction. 

Pan 

Allows  the  entire  scene  to  be  rotated 
around  the  users  X  and  Y  axis 

Additional  movement  through  the  use  of  coded  Viewpoints  is  also  provided. 
Viewpoints  allow  an  analyst  to  quickly  move  to  an  area  of  interest  with  a  click  of  the 
mouse  or  a  limited  sequence  of  keystrokes.  A  segment  of  Viewpoint  code  is  shown 
here: 
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DEF  ViewPointl  Viewpoint  { 
description  "ViewPointl " 
jump  TRUE 
fieldOJView  0.79 
position  0.0  0.0  0.0 

} 

This  segment  defines  an  X,  Y,  Z  position  relative  to  the  current  coordinate  system  and 
a  field  of  view,  in  radians,  indicating  the  spread  angle  of  the  viewpoint’s  viewing- 
volume  frustum.  Additionally,  the  jump  being  set  to  TRUE  ensures  the  new 
viewpoint  becomes  the  current  user  view. 

The  Blaxxun  3D  controls  coupled  with  Viewpoints  allow  a  user  to  gracefully 
move  through  BGONE,  observing  the  various  products  and  data  elements  presented. 
However,  navigation  of  the  visual  environment  provides  only  a  partial  solution.  Text 
handling  must  also  be  integrated. 

5.3.2  Textual  Navigation 

Once  a  desired  product  is  visually  selected  for  exploitation,  the  user  must  have 
effective  text  handling  techniques  to  benefit  from  the  initial  data  visualization.  As 
seen  in  figure  5.14,  integrating  a  text  box  with  a  VRML  environment  is  conducive  to 
the  use  of  frames.  Project  Broadsword  already  utilizes  frames  so  their  inclusion  was 
seamless. 

Text  handling  through  the  use  of  frames  allows  the  analyst  to  maintain  the 
visual  results  while  examining  the  textual  details.  This  is  an  improvement  over  a  pure 
text  approach,  but  more  advanced  methods  through  dialog  boxes  were  explored. 

Natively,  the  VRML  language  provides  a  function  termed  TouchSensor  that 
enables  animation  based  on  mouse  interactions.  For  example,  placing  a  mouse  cursor 
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over  a  sphere  and  pressing  the  left  mouse  button  can  cause  the  sphere  to  spin  on  its 
axis.  This  interaction  is  an  ideal  candidate  for  the  support  of  pop-up  dialog  boxes 
containing  intelligence  product  data.  Examination  and  experimentation  with  these 
touchsensors  revealed  VRML  is  not  capable  of  providing  context-sensitive  dialog 
boxes.  A  supporting  language,  such  as  JavaScript,  was  required.  JavaScript  is  well 
situated  for  browser-based  text  handling  tasks.  However,  being  designed  to  co¬ 
habitat  with  HTML,  it  does  not  readily  support  the  VRML  environment. 
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For  this  research,  methods  were  created  for  the  control  and  generation  of  data 
elements  when  interacting  with  glyphs.  These  new  methods  combined  the  animation 
functions  of  VRML  with  the  text  handling  capabilities  of  JavaScript.  The  resulting 
functionality  enables  BGONE  to  meet  the  needs  of  Project  Broadsword  by  providing 
quick  and  efficient  access  to  data  elements  not  mapped  to  a  glyph  attributes. 

These  new  methods  provide  context-sensitive  dialog  boxes  and  data  handling, 
through  three  VRML  interactions:  isActive,  isPressed  and  isOver.  Figure  5.15  shows 
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Figure  5.15:  Pop-up  Dialog  Box 

a  resulting  window  when  the  isActive  method  is  called.  This  method  opens  a  dialog 
box  whenever  a  mouse  button  is  pressed  and  held  while  over  a  glyph.  When  the 
mouse  button  is  released  the  dialog  box  closes.  Figure  5.16  demonstrates  the 
isPressed  method  that  generates  the  associate  data,  in  this  case  an  F-15E,  in  the  right 
frame  of  the  browser  window.  This  is  accomplished  by  placing  the  mouse  cursor 
over  a  glyph  and  single  clicking  the  mouse.  Figure  5.16  also  shows  activation  of  the 
isOver  method.  This  method  opens  a  dialog  box  whenever  the  mouse  cursor  is 
moved  over  a  glyph.  The  box  remains  open  until  the  cursor  is  moved  completely  off 
of  the  glyph.  The  automatic  opening  and  closing  of  the  dialog  box  provides  a  very 
quick  glance  capability  as  the  mouse  cursor  is  moved  from  one  product  to  the  next. 
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F-15E  Strike  Eagle 


Mission 

The  F-15E  Strike  Eagle  is  a  dual -role  fighter  designed  to  perform 
air-to-air  and  air-to-ground  missions.  An  array  of  avionics  and 
electronics  systems  gives  the  F-15E  the  capability  to  fight  at  low 
altitude,  day  or  night,  and  in  inclement  weather  and  perform  its 
primary  function  as  an  air-to-ground  attack  aircraft 

Features 

The  aircraft  uses  two  crew  members,  a  pilot  and  a  weapon 
systems  officer.  Previous  models  of  the  F-15  are  assigned 
air-to-air  roles;  the  "E"  model  is  a  dual-role  fighter.  It  has  the 
capability  to  fight  its  way  to  a  target  over  long  ranges,  destroy 
enemy  ground  positions  and  fight  its  way  out 

An  inertial  navigation  system  uses  a  laser  gyro  to  continuously 
monitor  the  aircraft’s  position  and  provide  information  to  the 
central  computer  and  other  systems,  including  a  digital  moving 
map  in  both  cockpits. 

The  APG-70  radar  system  allows  air  crews  to  detect  ground 
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Data  placed  in  frame  via 
the  isPressed  method. 


Figure  5.16:  Text  Handling  (Color  Plate  17) 
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5.4  Multimedia 


To  expand  the  data  exploitation  capabilities  within  BGONE,  multimedia 
concepts  were  explored.  Specifically  targeted  was  the  integration  of  streaming  video. 
To  enhance  exploitation  of  this  medium,  the  inner  glyph  was  cast  as  a  cube  with  the 
video  stream  texture  mapped  onto  it.  Video  support  is  provided  through  the  use  of  the 
Real  Audio  player  plugin.  Playing  of  the  video  is  triggered  by  activation  of  a 
touchsensor. 

5.5  VRML  Plugin  Verification 

Operation  of  BGONE  requires  a  VRML  plugin  be  present.  Detection 
software  was  created  to  prevent  program  execution  if  a  plugin  is  not  installed.  This 
new  code  identifies  which  browser  is  being  used  and  then  checks  for  known  x86 
based  VRML  plugins.  These  include  Blaxxun  3D,  Contact,  Cosmo  Player,  Microsoft 
VRML  2.0  Viewer,  Viscape  Universal  and  Worldview.  If  a  plugin  is  detected  the 
user  notices  no  difference  during  startup.  If  not,  an  informational  message  is 
displayed. 

5.6  Conclusion 

This  chapter  focuses  on  the  design  approach  and  obstacles  encountered  during 
the  implementation  of  BGONE.  It  was  clearly  evident  early  on  in  the  research  that 
design  tradeoffs  would  have  to  be  made  in  order  to  mirror  the  security  and  integrity 
mandates  placed  on  Project  Broadsword.  The  creation  of  new,  never  before  seen, 
data  handling  methods  gives  BGONE  a  unique  operating  base  on  which  to  explore 
further  data  visualization  and  text  handling  capabilities. 
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6.  Findings 


6.1  Introduction 

The  primary  objective  of  this  thesis  is  to  show  that  three-dimensional  data 
visualization  can  provide  an  analyst  a  more  efficient  and  effective  way  of  organizing, 
understanding  and  navigating  vast  amounts  of  intelligence  products.  This 
visualization  approach  is  specifically  tailored  to  Project  Broadsword,  an  intelligence 
data  retrieval  system,  being  developed  at  the  Air  Force  Research  Laboratory/Rome 
Research  Site  in  Rome,  New  York.  The  creation  of  the  Broadsword  Geospatial 
Oriented  Navigational  Environment  (BGONE),  a  stand-alone  representative 
environment  of  Project  Broadsword,  provided  a  tightly  controlled  test  environment 
through  which  the  Project  Broadsword  verification  and  validation  team  could 
compare  their  current  intelligence  exploitation  methods  to  those  provided  by 
BGONE.  The  findings  presented  here  are  based  on  results  while  employing  the  test 
scenario  outlined  in  Section  3.5.  For  the  remainder  of  this  chapter,  the  Project 
Broadsword  verification  and  validation  team  will  be  referred  to  as  ‘testers’. 

6.2  Experiment  Setup 

Intelligence  products  used  in  this  research  were  obtained  from  unclassified 
sources  located  inside  the  Project  Broadsword  development,  testing  and  integration 
vault.  Three  sources  were  utilized;  the  Military  Intelligence  Database  (MIDB),  the 
Automated  Message  Handling  System  (AMHS),  and  the  Image  Product 
Library/Image  Product  Archive  (IPL/IPA).  These  sources  provided  products  from  all 
four  types;  audio,  video,  imagery,  and  text  documents. 
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Following  the  guidelines  presented  in  Section  3.5,  testers  performed  each  task 
three  times.  First,  each  scenario  was  completed  using  the  live  Project  Broadsword 
system  in  ‘text  results  only’  mode.  Next,  the  scenario  was  accomplished  with  the 
system  in  ‘geospatial  map  results’  mode.  Finally,  the  scenario  was  exercised  using 
BGONE. 

6.3  Experiment  Results 

Experiment  results  are  based  on  the  combined  responses  received  during  tester 
interviews  upon  completion  of  each  scenario.  Each  tester  completed  a  single 
experiment  cycle  (the  scenario  exercised  in  each  of  the  three  configurations  as 
described  above)  once. 

6.3.1  Visual  Presentation 

People  have  a  natural  acceptance  and  appeal  for  visual  stimuli.  As  discussed 
in  Chapter  2,  the  average  individual  can  recognize  and  comprehend  a  complete 
graphical  image  much  faster  than  reading  a  single  line  of  text.  BGONE  taps  into  this 
acceptance  by  providing  an  environment  where  cognitive  functions  are  stimulated 
through  visual  depictions  of  intelligence  products.  Embracing  this  phenomenon, 
testers  found  themselves  naturally  drawn  to  the  scenario  results  provided  by  BGONE. 
Although  not  a  technical  measure,  they  found  the  rich  color  and  sense  of  depth  caused 
by  using  three-dimensional  objects  pleasing  to  the  eye.  Compared  to  the  existing 
two-dimensional  approach,  testers  felt  that  BGONE  portrayed  an  enhanced  realism 
that  allowed  them  to  categorize  and  explore  geographically  based  intelligence  data 
more  quickly.  The  most  prominent  factor  to  this  perceived  speedup  was  the  testers’ 
ability  to  more  easily  comprehend  the  overall  clustering  of  products  within  the  area  of 
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interest.  Testers  also  commented  that  the  ability  to  alter  one’s  viewpoint  without 
redrawing  the  scene,  as  is  currently  done  in  the  two-dimensional  geospatial  map, 
allowed  for  a  more  fluid  transition  when  directing  attention  from  one  product  to  the 
next.  In  response  to  the  added  visual  intensity  of  utilizing  three-dimensional  space, 
testers  overwhelmingly  conveyed  that  its  use  enhanced  the  data  exploitation 
experience.  Their  observations  were  based  on  the  fact  that  the  increase  in  the  number 
of  products  able  to  be  viewed  at  a  single  time  is  a  vast  improvement  over  the  current 
two-dimensional  map  and  textual  based  display.  The  ability  to  view  a  large  number 
of  products  in  a  given  instance  of  time  increases  efficiency  by  reducing  the  amount  of 
time  and  user  intervention  required  to  browse  through  all  products  of  interest. 

Besides  visual  acceptance,  testers  examined  whether  BGONE  increased 
understanding  of  the  total  number  of  products  available  when  compared  to  the  two- 
dimensional  map.  Testers  found  that  BGONE  increased  product  results  knowledge 
by  enabling  a  better  understanding  of  the  number  of  products  located  at  a  given 
location.  This  is  accomplished  in  two  ways.  First,  each  individual  product  is 
represented  and  displayed  as  a  single  glyph.  However,  when  the  number  of  products 
available  at  single  location  exceeds  a  user-specified  parameter  or  the  available  scene 
space  is  depleted,  a  gray  glyph  takes  on  the  representation  of  multiple  products.  For 
example,  with  a  gray  glyph  set  to  represent  five  products,  Figure  6.1  displays  seven 
available  products. 

The  two-dimensional  map,  seen  in  Figure  6.2,  uses  a  single  green  square  to 
represent  all  products  at  a  given  location.  There  is  no  mechanism  to  enable  the  user 
to  gain  an  understanding  of  the  number  or  types  of  products  available.  This 


53 


Figure  6.1:  Glyph  Representation  (Color  Plate  18) 


Single  Green  Square 


Figure  6.2:  Two-dimensional  Geospatial  Map  (Color  Plate  19) 


understanding  requires  the  icons  to  first  be  explored.  Exploring  an  icon  requires  the 
opening  of  a  new  window  and  returning  to  a  text  based  format.  Testers  found  this 
method  of  operation  to  be  counter  productive. 

Beyond  the  number  of  products,  testers  preferred  the  BGONE  method  of 
mapping  product  types  to  the  color  of  the  outer  glyph.  When  performing  scenario 
task  one,  the  ability  to  quickly  identify  textual  products  by  the  color  purple  made  for 
light  work.  Presented  with  the  two-dimensional  map,  task  one  required  the 
exploration  of  every  icon  coupled  with  a  text  based  visual  search  for  products  marked 
as  ‘text’.  Using  BGONE,  gray  glyphs  also  had  to  be  explored;  however,  they  did  not 
require  textual  searching.  Overall  each  tester  performed  task  one  in  approximately 
fifty  percent  less  time  when  utilizing  BGONE. 

6.3.2  Visual  Navigation 

Three-dimensional  data  visualization  over  a  geospatial  map  may  require  a  user 
to  physically  navigate  the  virtual  space.  Since  task  one,  locating  the  total  number  of 
textual  products,  only  required  identification  of  the  outer  glyph,  it  could  be 
accomplished  using  only  low  level  of  detail  glyphs  (as  described  in  Section  5.2.3). 
Glyphs  can  be  selected  at  any  visible  distance,  therefore,  no  movement  within  the 
virtual  world  was  required.  Task  two,  retrieving  all  imagery  products  that  have  an 
IPL  source,  forced  testers  to  move  about  the  virtual  landscape  tripping  proximity 
sensor  to  reveal  the  inner  model.  Testers  found  this  movement  difficult  and  time 
consuming.  The  most  common  problem  was  loosing  track  of  their  current  virtual 
location.  Additionally,  maintaining  the  correct  visual  perspective  required  careful 
movements  to  prevent  disorientation.  When  compared  with  the  textual  and  two- 
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dimensional  map  based  approach,  performing  task  two  with  BGONE  took 
approximately  four  times  longer  to  complete.  Prior  to  this  experiment,  most  users 
had  little  to  no  experience  navigating  in  a  three-dimensional  environment. 

6.3.3  Textual  Data  Navigation 

Task  three,  accounting  for  all  F15-E  imagery  products,  built  on  the 
requirements  of  task  two  by  adding  the  additional  need  to  locate  more  data  than  can 
be  visually  displayed.  Testers  had  the  same  navigational  experiences  as  described  in 
Section  6.3.2,  however,  once  there  they  had  to  contend  with  locating  the  correct 
information.  When  fulfilling  task  three  using  the  textual  and  two-dimensional  map 
approach,  testers  searched  through  a  laundry  list  of  products  to  locate  the  required  F- 
15E  imagery.  As  shown  in  Figure  6.3,  using  frames  and  automatic  pop-up  dialog 
boxes,  as  described  in  Chapter  5,  BGONE  enabled  testers  to  more  efficiently  locate 
the  needed  information.  This  is  made  possible  because  a  dialog  box  provides  a 
quick-look  capability  for  key  elements  of  information.  Figure  6.3,  therefore,  enables 
us  to  quickly  view  the  product  type,  resolution  of  the  image,  the  data  source,  and  the 
originating  gatekeeper  without  having  to  sift  through  all  the  data  available.  Although 
the  testers  highly  praised  the  text  handling  of  BGONE,  getting  to  that  point  in  the 
experiment  caused  task  three  to  take  five  times  longer  to  complete  versus  using  the 
textual  or  two-dimensional  map  methods. 
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F-15E  Strike  Eagle 


Mission 

The  F-15E  Strike  Eagle  is  a  dual-role  fighter  designed  to  perform 
air-to-air  and  air-to-ground  missions.  An  array  of  avionics  and 
electronics  systems  gives  the  F-15E  die  capability  to  fight  at  low 
altitude,  day  or  night,  and  in  inclement  weather  and  perform  its 
primary  function  as  an  air-to-ground  attack  aircraft 

Features 

Ik  aircraft  uses  two  crew  members,  a  pilot  and  a  weapon 
systems  officer.  Previous  models  of  the  F-15  are  assigned 
air-to-air  roles;  the  "E"  model  is  a  dual-role  fighter.  It  has  the 
capability  to  fight  its  way  to  a  target  over  long  ranges,  destroy 
enemy  ground  positions  and  fight  its  way  out 

An  inertial  navigation  system  uses  a  laser  gyro  to  continuously 
monitor  the  aircraft’s  position  and  provide  information  to  the 
central  computer  and  other  systems,  including  a  digital  moving 
map  in  both  cockpits. 

The  APG-70  radar  system  allows  air  crews  to  detect  ground 
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Figure  6.3:  Product  Exploitation  with  BGONE 
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6.4  Conclusion 


These  findings  paint  a  mixed  picture,  depending  on  which  issues  you  place  the 
most  emphasis.  Data  visualization  in  three-dimensional  space  appears  to  bring  new 
opportunities  with  challenges  to  match.  Despite  the  challenges,  testers  found 
BGONE  to  be  a  solid  basis  for  which  to  test  and  examine  Project  Broadsword  data 
visualization  issues.  Conclusions  and  recommendations  for  further  research  based  on 
these  findings  are  presented  in  Chapter  7. 
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7.  Conclusions  and  Recommendations 


7.1  Introduction 

This  research  set  out  to  explore  how  data  visualization  techniques  could 
enhance  an  intelligence  analyst’s  exploitation  capability.  Specifically  targeted  were 
Project  Broadsword  and  its  unique  ability  to  access  and  disseminate  data  from 
heterogeneous  sources  worldwide.  To  investigate  data  visualization  concepts  in 
controlled  experiments  required  construction  of  a  representative  modeled 
environment.  This  environment,  known  as  the  Broadsword  Geospatially  Oriented 
Navigational  Environment  (BGONE),  supplied  the  framework  and  tools  to  simulate 
how  an  adept  user  in  the  field  would  respond  and  navigate  a  geospatially  disperse 
field  of  data. 

This  chapter  discusses  implementation  considerations  based  on  the  findings  in 
Chapter  6.  Unique  contributions  to  the  field,  developed  as  a  result  of  this  research, 
are  also  covered.  Finally,  recommendations  for  future  research  are  presented. 

7.2  Conclusions 

The  experiments  run  in  BGONE  offered  a  wealth  of  information.  The  most 
notable  being  that  the  primary  concern  with  data  visualization  in  a  three-dimensional 
space  is  not  the  visuals,  but  rather  the  navigation  required  moving  from  one  product 
to  the  next.  Given  that  our  dataset  is  geospatially  placed  on  a  worldwide  scale,  the 
amount  of  virtual  terrain  that  must  be  traversed  may  be  quite  large.  Using  only 
standard  two-dimensional  input  devices,  a  mouse  and  keyboard,  most  testers  who  had 
to  travel  more  than  a  few  virtual  meters  found  the  experience  quite  frustrating. 

The  intelligence  data  products  used  for  this  research  contain  a  large  number  of 
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data  elements.  This  large  quantity  is  more  than  purely  visual  methods  can  effectively 
display.  The  addition  of  pop-up  dialog  boxes  effectively  aided  testers  in  the 
completion  of  experiment  tasks  requiring  exploitation  of  specific  data  elements  held 
by  a  product.  Unlike  the  two-dimensional  map  method,  testers  were  able  to  quickly 
move  from  one  product  to  the  next  without  returning  to  a  text-based  laundry  list  of 
products. 

This  research  focused  around  the  display  of  representative  geometric  objects 
in  three-dimensional  space.  Data  visualization  is  a  powerful  tool  that  becomes 
exponentially  more  complicated  to  develop  and  control  when  a  user  must  physically 
interact  with  the  data  they  are  viewing.  Those  who  have  little  to  no  experience  with 
three-dimensional  environments  may  find  this  type  of  data  visualization 
uncomfortable  to  work  with.  As  discussed  in  section  6.3.2,  testers  had  a  difficult  time 
navigating  within  BGONE.  Had  this  not  been  a  research  experiment,  most  agreed 
they  would  have  reverted  to  a  text  based  data  exploitation  method  to  “get  the  job  done 
quickly”.  The  use  and  implementation  of  knowledge  and  techniques  developed  from 
this  study  must  therefore  complement,  not  replace,  current  intelligence  exploitation 
methods.  As  with  any  updated  technology  or  method  of  doing  business,  people  need 
time  to  conquer  the  learning  curve  before  benefits  can  more  accurately  be  assessed. 
7.2.1  Implementation  Consideration 

The  Project  Broadsword  program  office,  encouraged  strongly  by  the  user 
community,  is  looking  to  expand  into  the  three-dimensional  realm  when  using  data 
visualization  methods.  Although  they  wish  to  give  the  users  what  they  want,  I  feel 
that  jumping  headfirst  into  such  an  undertaking  is  a  mistake.  At  this  time,  the 
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maturity  level  of  the  technology  and  standard  computer  input  tools  supplied  to  each 
analyst  are  not  yet  sufficient.  Rather  than  a  full  implementation  of  three-dimensional 
data  visualization,  as  presented,  a  gradual  integration  of  this  technology  should  be 
explored.  One  possible  technique  involves  coupling  results  from  the  current  two- 
dimensional  geospatial  map  with  BGONE  enabled  exploitation.  When  an  analyst  is 
presented  with  results,  as  seen  in  Figure  7.1,  rather  than  the  green  icon  leading  to  a 


Figure  7.1:  Current  Results  Display  (Color  Plate  20) 

text-based  format,  a  new  window  such  as  Figure  7.2  could  be  displayed.  This  new 
window  shows  all  products  available  at  the  chosen  location  of  interest.  With  the 
results  map  left  in  an  unmodified  window,  the  analysts  can  visually  explore  the 
locality  without  losing  global  perspective.  Applying  the  concept  of  Figure  7.2,  the 
new  data  navigational  techniques  allow  for  quick  exploitation  of  products  of  interest 
without  physical  movement  within  the  three-dimensional  space.  This  provides 
improved  intelligence  exploitation  techniques,  as  discussed  in  Section  7.2,  while 
alleviating  the  discomfort  testers  felt  while  traversing  a  VRML  map.  This 
research  set  out  to  develop  new  data  exploitation  methods  and  techniques  for  the 
plethora  of  digital  intelligence  products.  Implementing  the  data  navigation 
techniques,  as  described  in  Section  5.3.2,  required  the  creation  of  new  methods  that 
allow  automatic  pop-up  dialog  box  generation  in  a  VRML  environment.  This 
capability,  never  before  seen  in  this  format,  is  crucial  for  the  effective  use  of  data 
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Figure  7.2:  BGONE  Enhanced  Results  (Color  Plate  21) 


visualization  when  the  amount  of  data  to  be  exploited  exceeds  the  handling 
capabilities  of  the  graphics.  The  application  of  these  techniques  can  and  should  be 
applied  to  other  data  visualization  problems  where  a  large  amount  of  information  is 
present.  These  other  problems  include  data  exploration  in  a  two-dimensional 
environment,  which  highlight  the  fact  that  the  creative  handling  of  data  in  two- 
dimensional  visual  space  has  not  yet  been  exhausted. 

7.3  Further  Research 

This  thesis  set  out  to  demonstrate  the  benefits  of  exploiting  intelligence 
products  using  data  visualization  employed  in  three-dimensional  space.  However, 
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research  revealed  that  more  work  is  needed  before  successful  implementation  in  this 
area.  Numerous  areas  of  study  need  to  be  pursued,  to  include  correlation  of  a 
products’  representative  size  to  the  users’  view  level,  virtual  reality  device  usage,  and 
enhanced  multimedia  capabilities. 

Under  the  current  architecture,  proper  scaling  between  a  glyph  or  an  icon  with 
respect  to  its  location  on  the  geospatial  map  is  not  performed.  As  a  user  moves  about 
the  map  or  zooms  in  and  out,  the  icon  size  does  not  change  with  respect  to  the  users’ 
view.  For  example,  at  one  level  of  resolution  a  single  result’s  icon  can  encompass 
fifty  square  nautical  miles  of  area,  while  in  another  the  same  results  icon  may  cover 
only  one  square  mile.  Although  the  glyph  size  does  change  with  respect  to  a  users’ 
view,  the  geospatial  location  over  which  it  is  located  is  not  defined  in  exact  detail 
since  this  information  is  not  passed  to  the  graphics  Tenderer.  The  design  process 
currently  arbitrarily  sets  glyph  and  icon  sizes.  A  method  for  properly  controlling  and 
varying  the  size  of  the  glyph  or  icon  with  respect  to  the  actual  data  location  as  the 
analyst  changes  worldview  perspective  needs  to  be  addressed. 

Since  most  analysts  are  constrained  physically  by  two  dimensional  control 
devices,  effective  ways  of  breaking  out  of  this  confinement  need  to  be  explored. 

Using  virtual  reality  devices,  such  as  a  space  ball  or  head  tracker,  would  allow 
additional  freedom  and  ease  of  movement.  Inclusion  of  these  devices  may  effectively 
eliminate  the  movement  difficulties  outlined  in  Section  6.3.2. 

This  research  has  only  brushed  the  surface  of  what  can  be  achieved  with 
multimedia.  As  network  bandwidth  and  computer  processing  increases,  the 
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possibilities  for  video  streaming  (both  live  and  archived)  and  other  technologies  hold 
many  promises  for  future  enhancements. 
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Color  Plate  2:  Preattentive  Processing 


Color  Plate  3:  Symbolic  Glyphs 
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Color  Plate  4:  Multilevel  Glyphs 


Color  Plate  5:  Cylinder  Shaped  Level  One  Glyphs 
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Color  Plate  6:  Cube  Shaped  Level  One  Glyphs 


Color  Plate  7:  Transparency  as  Designed 
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Color  Plate  8:  Transparency  in  Operational  Environment 


Color  Plate  9:  Redesigned  Level  One  Glyph 
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Color  Plate  10:  Abstract  Level  Two  Glyph 


Color  Plate  11:  Model  Based  Level  Two  Glyph 
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Color  Plate  12:  Low  Level  Of  Detail  Glyphs 


Color  Plate  13:  Low  Level  Of  Detail  Glyphs  on  a  VRML  Landscape  Map 
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Color  Plate  14:  Proximity  Sensor  Activation  Rendering 
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Color  Plate  15:  Products  Represented  by  Gray  Glyph 
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Color  Plate  16:  VRML  Environment  with  Frames 
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Color  Plate  17:  Text  Handling 
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Color  Plate  18:  Glyph  Representation 
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Color  Plate  19:  Two-dimensional  Geospatial  Map 


Color  Plate  20:  Current  Results  Display 
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Color  Plate  21:  BGONE  Enhanced  Results 
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